Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.

The security of artificial intelligence (AI) is an important research area towards safe, reliable, and trustworthy AI systems. To accelerate the research on AI security, the Artificial Intelligence Security Competition (AISC) was organized by the Zhongguancun Laboratory, China Industrial Control Systems Cyber Emergency Response Team, Institute for Artificial Intelligence, Tsinghua University, and RealAI as part of the Zhongguancun International Frontier Technology Innovation Competition (https://www.zgc-aisc.com/en). The competition consists of three tracks, including Deepfake Security Competition, Autonomous Driving Security Competition, and Face Recognition Security Competition. This report will introduce the competition rules of these three tracks and the solutions of top-ranking teams in each track.

In this paper, we consider incorporating data associated with the sun's north and south polar field strengths to improve solar flare prediction performance using machine learning models. When used to supplement local data from active regions on the photospheric magnetic field of the sun, the polar field data provides global information to the predictor. While such global features have been previously proposed for predicting the next solar cycle's intensity, in this paper we propose using them to help classify individual solar flares. We conduct experiments using HMI data employing four different machine learning algorithms that can exploit polar field information. Additionally, we propose a novel probabilistic mixture of experts model that can simply and effectively incorporate polar field data and provide on-par prediction performance with state-of-the-art solar flare prediction algorithms such as the Recurrent Neural Network (RNN). Our experimental results indicate the usefulness of the polar field data for solar flare prediction, which can improve Heidke Skill Score (HSS2) by as much as 10.1%.

We propose LiDAL, a novel active learning method for 3D LiDAR semantic segmentation by exploiting inter-frame uncertainty among LiDAR frames. Our core idea is that a well-trained model should generate robust results irrespective of viewpoints for scene scanning and thus the inconsistencies in model predictions across frames provide a very reliable measure of uncertainty for active sample selection. To implement this uncertainty measure, we introduce new inter-frame divergence and entropy formulations, which serve as the metrics for active selection. Moreover, we demonstrate additional performance gains by predicting and incorporating pseudo-labels, which are also selected using the proposed inter-frame uncertainty measure. Experimental results validate the effectiveness of LiDAL: we achieve 95% of the performance of fully supervised learning with less than 5% of annotations on the SemanticKITTI and nuScenes datasets, outperforming state-of-the-art active learning methods. Code release: https://github.com/hzykent/LiDAL.

Speech representation learning has improved both speech understanding and speech synthesis tasks for single language. However, its ability in cross-lingual scenarios has not been explored. In this paper, we extend the pretraining method for cross-lingual multi-speaker speech synthesis tasks, including cross-lingual multi-speaker voice cloning and cross-lingual multi-speaker speech editing. We propose a speech-text joint pretraining framework, where we randomly mask the spectrogram and the phonemes given a speech example and its transcription. By learning to reconstruct the masked parts of the input in different languages, our model shows great improvements over speaker-embedding-based multi-speaker TTS methods. Moreover, our framework is end-to-end for both the training and the inference without any finetuning effort. In cross-lingual multi-speaker voice cloning and cross-lingual multi-speaker speech editing tasks, our experiments show that our model outperforms speaker-embedding-based multi-speaker TTS methods. The code and model are publicly available at PaddleSpeech.

最近，神经技术已用于自动生成源代码。这些方法在有望获得声明语言的同时，在命令式语言的数据集上的性能差得多。由于通常将声明性语言嵌入了现实世界软件开发中的命令式语言（即Turducken式编程）中，因此声明语言的有希望的结果几乎不会导致手动软件开发工作大幅减少。在本文中，我们定义了一项新的代码生成任务：鉴于自然语言评论，此任务旨在用嵌入式声明语言以基本命令性语言生成程序。据我们所知，这是第一个Turducken风格的代码生成任务。对于此任务，我们将Lyra：Python中的数据集提出了嵌入式SQL。该数据集包含来自现实世界项目的2,000个精心注释的数据库操作程序。每个程序都与中文评论和英文评论配对。在我们的实验中，我们采用了变压器，伯特风格和GPT风格的模型作为基础。在最佳环境中，GPT风格模型的生成性能比其他模型更好，在使用中文和英语评论时，AST精确匹配的精度分别为24％和25.5％。因此，我们认为Lyra为代码生成提供了新的挑战。但是，克服这一挑战可能会大大提高代码生成技术在现实世界软件开发中的适用性。

近年来，由于强大的3D CNN，基于体素的方法已成为室内场景3D语义分割的最新方法。然而，基于体素的方法忽略了基础的几何形状，由于缺乏地理位置信息而在空间上闭合物体上的模棱两可的特征遭受了含糊的特征，并努力处理复杂和不规则的几何形状。鉴于此，我们提出了Voxel-Mesh网络（VMNET），这是一种新颖的3D深度体系结构，该架构在Voxel和网格表示上运行，并利用了欧几里得和地球信息。从直觉上讲，从体素中提取的欧几里得信息可以提供代表附近对象之间交互的上下文提示，而从网格中提取的地理信息可以帮助空间上接近但断开表面的分离对象。为了合并两个域中的此类信息，我们设计了一个内域的专注模块，以进行有效的特征聚集和一个用于自适应特征融合的专注于域间的模块。实验结果验证了VMNET的有效性：具体而言，在具有挑战性的扫描仪数据集上，用于大规模的室内场景分割，它的表现优于最先进的Sparseconvnet和Minkowskownet（74.6％vs 72.5％和73.6％）更简单的网络结构（17m vs 30m和38m参数）。代码发布：https：//github.com/hzykent/vmnet

现有图形神经网络（GNNS）很大程度上依赖于节点嵌入品，其表示节点作为其标识，类型或内容的矢量。但是，具有未分配的节点的图表广泛存在于现实世界中的应用程序（例如，匿名社交网络）。以前的GNN可以将随机标签分配给节点（将伪影介绍给GNN）或分配给所有节点的一个嵌入（这不能明确区分一个节点）。此外，当这些GNN应用于未分配的节点分类问题时，它们具有不需要的标准性属性，其基本上无法以多种可能的输出来解决数据。在本文中，我们分析了节点分类问题现有方法的限制。灵感来自我们的分析，我们提出了一种推广的标准性质和优先标记技术，满足所需的属性渐近。实验结果表明，我们在几种未分配的节点分类任务中实现了高性能。

People with diabetes are more likely to develop diabetic retinopathy (DR) than healthy people. However, DR is the leading cause of blindness. At present, the diagnosis of diabetic retinopathy mainly relies on the experienced clinician to recognize the fine features in color fundus images. This is a time-consuming task. Therefore, in this paper, to promote the development of UW-OCTA DR automatic detection, we propose a novel semi-supervised semantic segmentation method for UW-OCTA DR image grade assessment. This method, first, uses the MAE algorithm to perform semi-supervised pre-training on the UW-OCTA DR grade assessment dataset to mine the supervised information in the UW-OCTA images, thereby alleviating the need for labeled data. Secondly, to more fully mine the lesion features of each region in the UW-OCTA image, this paper constructs a cross-algorithm ensemble DR tissue segmentation algorithm by deploying three algorithms with different visual feature processing strategies. The algorithm contains three sub-algorithms, namely pre-trained MAE, ConvNeXt, and SegFormer. Based on the initials of these three sub-algorithms, the algorithm can be named MCS-DRNet. Finally, we use the MCS-DRNet algorithm as an inspector to check and revise the results of the preliminary evaluation of the DR grade evaluation algorithm. The experimental results show that the mean dice similarity coefficient of MCS-DRNet v1 and v2 are 0.5161 and 0.5544, respectively. The quadratic weighted kappa of the DR grading evaluation is 0.7559. Our code will be released soon.

Zero-Shot Learning has been a highlighted research topic in both vision and language areas. Recently, most existing methods adopt structured knowledge information to model explicit correlations among categories and use deep graph convolutional network to propagate information between different categories. However, it is difficult to add new categories to existing structured knowledge graph, and deep graph convolutional network suffers from over-smoothing problem. In this paper, we provide a new semantic enhanced knowledge graph that contains both expert knowledge and categories semantic correlation. Our semantic enhanced knowledge graph can further enhance the correlations among categories and make it easy to absorb new categories. To propagate information on the knowledge graph, we propose a novel Residual Graph Convolutional Network (ResGCN), which can effectively alleviate the problem of over-smoothing. Experiments conducted on the widely used large-scale ImageNet-21K dataset and AWA2 dataset show the effectiveness of our method, and establish a new state-of-the-art on zero-shot learning. Moreover, our results on the large-scale ImageNet-21K with various feature extraction networks show that our method has better generalization and robustness.